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The  after  action  review  (AAR)  is  a  mechanism  for  providing  feedback  to  organizations  on  their  performance 
of  collective  tasks.  It  is  an  active  process  that  requires  unit  members  to  participate  in  order  to  benefit. 
The  AAR  is  a  method  of  providing  feedback  to  units  after  operational  missions  or  collective  training  exercises 
[23].  The  AAR  is  an  interactive  discussion,  guided  by  a  facilitator  or  trainer  known  as  an  AAR  leader.  During 
the  AAR,  unit  members  discuss  what  happened,  why  it  happened,  and  how  to  improve  or  sustain  performance 
in  similar  situations  in  the  future. 


1.0  SIMULATION-BASED  TRAINING  ENVIRONMENTS 

Military  Training  is  always  a  simulation  whether  it  occurs  at  a  sophisticated  instrumented  range,  in  a 
collective  training  simulator  system,  or  in  a  command  and  staff  exercise  driven  by  a  math  model  driven  war 
game.  Training  occurs  in  live,  virtual,  constructive,  or  mixed  simulations  of  battlefield  environments.  There 
are  always  compromises  in  training  with  how  tasks  would  be  performed  in  combat.  In  the  live  environment, 
units  use  operational  equipment  and  actual  terrain  and  perform  against  an  opposition  force  composed  of 
military  personnel  (live  force-on-force)  or  targets  (live  fire),  depending  upon  whether  the  unit  is  employing 
simulated  weapons’  effects  or  firing  live  rounds.  In  virtual  environments,  units  use  simulators  to  represent 
equipment  and  weapons.  Weapons  effects,  terrain  and  enemy  forces  are  computer  generated.  In  constructive 
environments,  battlefield  outcomes  (e.g.,  the  unit  lost  thirty  percent  of  its  personnel)  are  determined  by 
sophisticated  math  models  in  order  to  provide  battle  effects  supporting  command  and  staff  training.  Mixed 
environments  include  elements  of  two  or  more  of  the  simulation  environments.  Training  in  all  of  these 
simulation  environments  should  provide  individuals  and  units  with  feedback  about  how  their  actions 
contributed  to  mission  success  or  failure,  casualties  received,  and  casualties  inflicted  on  the  enemy,  the  bottom 
lines  of  collective  performance. 

Live  simulation  training  is  widely  available,  in  the  form  of  ranges  and  maneuver  areas.  The  most  highly 
supported  form  of  live  simulation  training  is  generally  found  at  ranges  that  support  engagement  simulation 
and  vehicle  or  aviation  asset  tracking.  Examples  of  these  ranges  are  the  US  Army’s  National  Training  Center 
or  the  US  Navy’s  Top  Gun  program.  They  differ  from  local  area  ranges  in  that  they  provide  a  cadre  of 
observer/controller/trainers,  a  dedicated  opposing  force,  instrumentation  that  is  capable  of  collecting  position 
location,  firing  and  status  data,  and  teams  of  analysts  supporting  observer/controller/trainers  from  a  data 
collection  and  analysis  facility. 

Virtual  Simulation  training  systems  vary  widely  across  services  and  nations.  Virtual  training  environments 
can  range  from  driver  and  gunnery  trainers  to  sophisticated  networks  of  simulators  representing  aviation 
assets  operating  in  a  coalition  format.  The  introduction  of  game-based  training  has  expanded  the  scope  of 
virtual  training  and  made  it  more  widely  available  because  of  the  games  relatively  low  cost. 

Constructive  Simulations  fall  into  two  categories.  Math  model  based  attrition  models  represent  the  actions  of 
military  units  and  serve  primarily  as  drivers  for  training  exercises  of  command  groups  and  staffs. 
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AFTER  ACTION  REVIEW  IN  SIMULATION-BASED  TRAINING 


The  constructive  simulation  provides  the  inputs  to  commanders  and  staffs  decision  making  processes. 
A  second  more  recent  form  of  constructive  simulation  is  more  entity  and  rule-based.  These  simulations 
provide  semi-automated  forces  to  be  enemy  and  adjacent  friendly  forces  for  virtual  simulations.  The  Close 
Combat  Tactical  Trainer  (CCTT),  a  U.S.  Army  heavy  forces  virtual  trainer,  utilizes  CCTT  SAP  to  generate 
enemy  forces  for  the  training  audience  to  fight. 

2.0  INTRINSIC  FEEDBACK 

Intrinsic  feedback  cues  and  guides  unit  behavior  during  task  performance  [4],  whether  the  task  is  being 
performed  in  an  operational  or  training  context..  For  example,  an  infantry  unit  may  call  in  artillery  fire  on  a 
target  and  receive  intrinsic  feedback  when  it  observes  that  simulated  or  actual  rounds  impact  too  far  from  the 
intended  target.  Someone  from  the  unit  assigned  to  observe  the  effects  of  the  artillery  fires  would  then  provide 
the  supporting  artillery  unit  with  guidance  for  adjusting  their  fires.  A  portion  of  intrinsic  feedback  comes  from 
simulations  or  actual  weapons  effects  (the  location  of  artillery  impact)  and  part  comes  from  unit  actions  (an 
observer  providing  the  artillery  unit  with  directions  for  adjusting  fires). 

As  a  rule  of  thumb,  there  are  more  gaps  in  intrinsic  feedback  for  a  unit  that  is  not  well  trained,  because  unit 
members  are  not  providing  their  portion  of  the  feedback.  Continuing  with  the  artillery  example,  not  having  an 
observer  in  position  can  result  in  a  gap  in  terms  of  a  unit’s  knowledge  of  the  effectiveness  of  its  artillery.  From 
a  broader  perspective,  if  there  are  failures  to  communicate  information  up  and  down  the  chain-of-command 
during  an  exercise  ,  then  there  will  be  gaps  in  feedback  needed  to  cue  and  guide  performance.  If  unit  members 
are  not  sure  about  what  aspects  of  the  tactical  situations  they  should  be  monitoring,  then  additional  gaps  in 
feedback  are  to  be  expected. 

An  important  difference  between  individual  and  collective  performance  is  that  in  collective  performance  much 
of  the  information  needed  to  cue  and  guide  performance  comes  from  other  people.  To  be  fully  trained  unit 
members  must  learn  how  to  provide  this  information  (i.e.,  their  part  of  the  intrinsic  feedback).  Improved 
capability  to  provide  intrinsic  feedback  at  the  right  time  to  the  right  people  is  evidence  that  unit  performance 
is  improving. 

3.0  EXTRINSIC  FEEDBACK 

For  a  unit  to  improve  its  performance  unit  members  will  in  most  cases  need  more  feedback  about  what 
happened  during  an  exercise  than  that  gained  by  participating  in  it  and  observing  what  happened.  Because  of 
the  so  called  “fog  of  war”,  when  an  exercise  is  over,  participants  sometimes  have  a  limited  perspective 
regarding  what  happened,  based  upon  the  information  available  to  them  and  what  they  saw,  heard  and 
smelled.  This  limited  perspective  is  referred  to  as  perceived  truth.  Ground  truth  is  the  term  used  to  describe 
the  actual  events  that  occurred.  Less  trained  units  are  expected  to  demonstrate  a  greater  disparity  between 
perceived  and  ground  truth,  simply  because  much  of  the  intrinsic  information  that  was  available  was  not  either 
perceived  or  used.  Events  may  be  happening  quickly  and  open  to  differing  interpretations.  Perceptions  and 
memories  of  the  occurrence,  sequence,  and  timing  of  events  can  be  greatly  distorted  leading  to  generation  of 
causal  relationships  which  are  not  based  on  the  actual  facts  [8]. 

Extrinsic  feedback  is  provided  by  an  outside  source,  usually  observer/controllers  or  trainers  after  an  exercise 
ends.  It  is  designed  to  help  participants  understand  the  ground  truth  situation  relative  to  their  perception  of 
perceived  truth  and  to  investigate  what  caused  the  events  to  occur  as  they  did.  Extrinsic  feedback  consists  of 
information  that  the  exercise  participants  don’t  ordinarily  have  available  to  them.  It  can  provide  insights  into 
how  to  improve  or  sustain  performance  in  the  future. 


8-2 


RTO-TR-HFM-121-Part-ll 


[NATO 

OTAISI 


AFTER  ACTION  REVIEW  IN  SIMULATION-BASED  TRAINING 


A  simulation  is  effective  to  the  extent  that  exercise  participants  can  appropriately  recognize  intrinsic  feedback 
regarding  their  performance,  and  extrinsic  is  provided  to  clarify  misperceptions.  Extrinsic  feedback, 
by  providing  information  on  exercise  outcomes,  allows  the  actions  of  individuals  to  be  linked  to  higher  level 
exercise  outcomes.  Sometimes  exercise  participants  recognize  the  impacts  of  their  actions  via  intrinsic 
feedback,  but  at  other  times  they  are  not  aware  of  these  impacts  until  they  receive  extrinsic  feedback. 


4.0  EXTRINSIC  FEEDBACK  METHODS  FOR  COLLECTIVE  TRAINING 

Formal  post-exercise  feedback  sessions  are  one  of  the  types  of  extrinsic  feedback  that  can  be  used  to  improve 
unit  performance  [4].  The  After  Action  Review  (AAR)  is  a  method  of  providing  extrinsic  feedback  to  units 
after  operational  missions  or  collective  training  exercises  [23].  Simply  put,  the  AAR  is  the  controlled  sharing 
of  intrinsic  feedback  combined  with  group  problem  solving.  Exercise  participants  play  differing  roles  and  are 
located  at  differing  points  within  the  battlespace,  so  each  participant  receives  relatively  unique  intrinsic 
feedback.  Extrinsic  feedback  can  be  used  to  correct  misperceptions  and  clarify  events  and  effects.  The  AAR 
process  may  provide  unit  members  with  a  view  of  collective  (team,  unit,  or  organizational)  performance  that 
was  not  apparent  to,  or  viewable  by,  any  one  participant  during  an  exercise  [14],  including  trainers  who  were 
observing  the  exercise.  The  AAR  uses  a  Socratic  Method  in  which  a  series  of  leading  and  open-ended 
questions  are  used  by  an  AAR  leader  to  help  those  in  the  training  audience  discover  what  happened  and  why. 

A  debrief  or  critique  conducted  by  one  or  more  observers  of  a  training  exercise  is  an  alternative  to  the  AAR 
[19,  9]  and  is  a  more  traditional  way  of  providing  feedback  by  trainers.  The  person  or  persons  who  provide  the 
critique  become  the  source  of  ground  truth  as  they  see  it.  Their  role  is  to  interpret  events  as  they  saw  them  and 
describe  to  the  training  participants  what  they  think  happened,  why  they  think  it  happened,  and  what  they 
think  the  unit  should  do  about  it.  Critiques  are  an  extrinsic  source  of  feedback.  A  major  difference  between  the 
AAR  and  critique  is  that  the  critique  provides  the  training  participants  with  conclusions  reached  by  the  person 
giving  the  critique  rather  than  facilitating  the  training  participants  to  reach  their  own  conclusions.  Critiques 
can  easily  be  taken  as  criticism  since  the  opinions  expressed  are  based  on  perceptions,  judgments  and  possibly 
misinterpretations  of  ground  truth.  Further,  the  critique  is  unable  to  make  use  of  diagnostic  information  that 
may  be  known  only  to  exercise  participants. 

The  AAR  leader  functions  as  a  discussion  facilitator.  Training  participants  are  expected  to  examine  their 
performance  through  guided  self  evaluation.  They  are  encouraged  to  identify  their  problems  and  develop 
approaches  to  correct  them.  It  is  assumed  that  use  of  the  AAR  feedback  method  results  in  units  claiming 
ownership  of  the  diagnosis  of  problems  and  the  corrective  actions  they  identify  [19]. 

Extrinsic  feedback  regarding  unit  performance  focuses  on  conceptual  knowledge  rather  than  procedural 
knowledge.  Feedback  is  likely  to  be  more  explanatory  than  directive  in  nature.  The  whole  process  of  using 
interactive  discussions  to  decide  what  happened,  why  it  happened,  and  how  to  improve  or  sustain  performance 
engenders  explanations.  Explanatory  feedback  is  superior  to  directive  feedback  in  terms  of  conceptual 
knowledge  acquisition  [16]. 

Post-exercise  feedback,  by  definition,  is  delayed  rather  than  immediate.  It  could,  conceivably  be  used  in 
conjunction  with  immediate  feedback  in  the  course  of  an  exercise  (i.e.,  through  coaching,  mentoring, 
intelligent  tutoring,  and/or  the  application  of  during  action  review  aids)  so  that  a  unit  can  take  immediate 
corrective  action  and  perhaps  accelerate  the  training  process  [11].  In  the  case  of  collective  training,  corrective 
actions  in  mid  exercise  may  help  prevent  a  unit  or  team  from  creating  a  tactical  situation  that  detracts  from  the 
intended  training  objectives  of  the  exercise. 
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5.0  HOW  REALISTIC  BATTLEFIELD  SIMULATIONS  SET  THE  STAGE  FOR  AN 
AAR 

The  AAR  was  based  upon  the  “interview  after  combat”  used  in  World  War  II  by  military  historian  Samuel 
Lyman  Atwood  (S.L.A.)  Marshall  and  others  to  find  out  what  happened  during  missions  [3].  The  process  was 
adapted  for  training  events  as  the  capability  to  provide  realistic  simulation  of  weapons  effects  occurred  during 
the  1970s  and  80s  [3,  17].  Prior  to  development  of  engagement  simulation  technologies  in  the  1970s  most 
military  collective  training  exercises  casualty  exchanges  and  mission  outcomes  were  based  upon  the 
subjective  judgments  of  umpires.  Such  judgments  were  insufficient  to  prepare  participants  for  an  “interview 
after  combat,”  because  the  participants  didn’t  believe  that  their  status  as  casualties  necessarily  resulted  from 
their  behavior.  The  development  of  tactical  engagement  simulation  technologies  provided  a  means  for 
objective  casualty  determination  [22].  Perhaps  the  best  known  example  of  TES  is  the  use  of  lasers  and  laser 
detectors  to  simulate  the  effects  of  line-of-sight  weapons,  such  as  rifles  and  tank  main  guns.  The  later 
development  of  virtual  simulations  such  as  Simulation  Networking  (SIMNET)  and  The  Close  Combat 
Tactical  Trainer  (CCTT)  eliminated  many  of  the  inaccuracies  of  live  TES  casualty  assessment  and  were 
capable  of  more  fully  representing  ground  truth  [7]. 


6.0  THE  ROLES  OF  AAR  AIDS  IN  SUPPORTING  FEEDBACK 

To  be  effective,  AAR  discussions  need  to  be  guided  by  an  AAR  leader.  The  leader  needs  one  or  more  start 
points  for  the  discussion  and  at  least  a  general  idea  of  where  the  discussion  will  head.  The  job  of  the  AAR 
leader  is  made  easier  to  the  extent  that  they  are  already  aware  of  the  types  of  problems  the  unit  has  been 
experiencing.  If  all  an  AAR  leader  knows  about  a  mission  is  that  a  unit  sustained  heavy  casualties, 
the  Socratic  Method  will  take  a  long  time  to  identify  the  root  causes  of  the  problem.  If  the  AAR  leader  and  the 
unit  know  that  most  of  the  casualties  occurred  within  a  few  minutes  of  making  contact  with  the  enemy  and 
that  few  friendly  vehicles  returned  fire  upon  contact,  they  are  closer  to  identifying  and  understanding  what 
happened  and  why.  The  AAR  does  not  require  an  exhaustive  review  of  all  aspects  of  a  unit’s  performance. 
Instead,  trainers  are  expected  to  focus  on  aspects  of  performance  closely  linked  to  key  exercise  events  and 
outcomes. 

At  instrumented  ranges  and  in  virtual  simulations  AAR  aids  prepared  from  electronic  data  streams  can 
document  or  illustrate  aspects  of  performance  that  are  close  to  root  causes  of  weaknesses  and  strengths. 
Developments  in  battlefield  simulations  technology  have  provided  trainers  with  a  record  of  electronic  data 
describing  position  location,  firing  events  and  communications  over  the  course  of  an  exercise.  AAR  software 
systems  have  been  developed  that  allow  this  data  to  be  converted  into  a  variety  of  AAR  aids  describing  or 
illustrating  ground  truth  [14].  For  example,  a  graph  showing  the  number  of  rounds  fired  by  each  vehicle  in  a 
platoon  over  time  may  make  the  point  that  only  one  of  the  vehicles  in  the  platoon  fire  was  involved  in  the 
early  portion  of  an  engagement.  To  gain  this  information  from  the  AAR  process,  a  unit  would  have  to  slowly 
reconstruct  the  sequence  of  events  based  on  their  memories.  AAR  aids  also  offer  the  benefit  of  providing  units 
with  demonstrable  ground  truth  when  their  recollections  are  at  odds  with  what  actually  happened. 

To  the  extent  that  AAR  aids  illustrate  root  causes  of  exercise  events,  rather  than  outcomes,  they  expedite  the 
AAR  process.  AAR  aid  generation  capabilities  that  examine  exercise  data  streams  to  check  specific  aspects  of 
performance  offer  a  means  of  helping  AAR  leaders  and  units  diagnose  strengths  and  weaknesses. 

The  most  frequently  used  AAR  aid  is  a  sequential  replay  of  exercise  events.  A  replay,  however,  is  not 
necessarily  the  most  efficient  or  effective  way  of  illustrating  key  aspects  of  performance.  Sometimes  AAR 
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aids  that  summarize  activity  over  a  period  of  time  can  be  more  effective.  A  graphic  tracing  the  movement  of  a 
unit  may  be  able  to  quickly  illustrate  that  a  unit  backtracked,  indicating  that  the  route  reconnaissance  may 
have  been  inadequate. 

Efforts  to  develop  innovative  AAR  aids  have  not  always  been  successful  [21].  Efficient  aids  should  provide 
information  that  would  otherwise  have  to  be  gleaned  from  lengthy  reviews  of  replays.  However,  poorly 
conceived  aids  tend  to  confuse  rather  than  clarify  the  situation  because  they  do  not  present  information  in  a 
manner  that  is  intuitive  or  clear  to  the  training  participants.  The  ideal  situation  would  be  one  in  which  training 
participants  have  learned  over  time  to  expect  certain  AAR  aids  to  be  presented  after  an  exercise  and  they 
know  what  information  is  to  be  gained  from  each  type  of  aid.  It  has  been  difficult  to  reach  this  goal  because 
each  training/simulation  environment  has  different  capabilities  to  support  AAR  aid  production.  Live 
simulations  are  limited  by  the  quality  of  the  data  generated  by  the  instrumentation  and  engagement  simulation 
systems.  Constructive  simulation  often  will  not  represent  entity  level  performance,  but  rather  aggregate 
performance  of  units.  As  mentioned  above,  virtual  simulations  represent  the  best  opportunity  to  provide 
accurate  and  appropriate  AAR  aids  because  most  of  the  data  of  interest  can  be  easily  collected  from  a 
network. 

AAR  aids  provide  units  with  an  improved  perspective  regarding  what  actually  happened  during  an  exercise 
that  more  accurately  reflects  ground  truth..  An  important  goal  of  the  unit  is  to  identify  corrective  actions  it  can 
take  that  will  provide  unit  members  with  an  improved  perspective  as  it  is  conducting  training  and  operations 
in  the  future  (i.e.,  better  intrinsic  feedback  to  cue  and  guide  unit  behavior). 


7.0  A  RECENTLY  DEVELOPED  AAR  TOOL  FOR  VIRTUAL  ENVIRONMENTS 

In  live  and  constructive  collective  training,  the  AAR  is  a  crucial  component  of  the  training  process.  The  same 
is  true  of  virtual  simulation  based  training.  The  first  collective  training  virtual  simulation,  SIMNET, 
was  developed  by  DARPA,  to  provide  training  and  better  understand  the  technical  requirements  of  networked 
simulators.  SIMNET  was  initially  developed  without  an  AAR  system,  a  capability  to  produce  AAR  aids  from 
recorded  movement,  communication  and  firing  data.  AARs  were  dependent  on  the  perceptions  of  the  training 
participants.  In  the  early  1990s  the  US  Army  Research  Institute  with  contracting  support  from  Perceptronics 
and  the  Institute  for  Simulation  and  Training  developed  the  Unit  Performance  Assessment  System  [13]  to 
capture  SIMNET  exercise  data  and  provide  AAR  aids  to  support  the  AAR  process.  Later  the  Automated 
Training  Analysis  and  Feedback  System  [4]  was  developed  also  for  the  SIMNET  system.  These  AAR  systems 
addressed  technical  issues  of  extracting  information  from  the  simulation  data  streams,  reducing  operator 
workload,  and  producing  aids  and  displays  that  went  far  beyond  a  simple  replay  of  the  exercise. 

In  the  1999  -  2002  timeframe,  as  part  of  an  overall  project  to  develop  capabilities  for  simulation-based 
training  of  dismounted  combatants,  an  AAR  system  was  developed  called  the  Dismounted  Infantry  Virtual 
AAR  System  (DIVAARS).  The  goal  was  to  develop  an  AAR  system  that  incorporated  lessons  learned  from 
earlier  AAR  systems  and  was  tailored  to  the  unique  requirements  of  small  unit  dismounted. 

Infantry  training  in  a  virtual  environment.  An  emphasis  was  placed  on  being  able  to  meet  the  special 
challenges  of  urban  environments  on  military  operations  and  training.  The  challenges  are  primarily  visual  in 
that  “buildings  and  other  structures  break  up  the  visual  field  and  limit  the  portion  of  the  battlefield  that  can  be 
observed  by  any  one  person.”  [12,  p.  59].  This  required  an  AAR  system  that  could  not  just  replay  an  exercise, 
but  could  in  addition  support  the  AAR  goals  of  presenting  exercise  events  and  data  in  a  manner  that  would 
facilitate  trainee  understanding  of  what  happened,  why  it  happened,  and  how  to  improve. 


RTO-TR-HFM-121-Part-ll 


8-5 


AFTER  ACTION  REVIEW  IN  SIMULATION-BASED  TRAINING 


ORGANIZATION 


For  determining  “what”  happened  during  a  mission,  the  DIVAARS  recreates  exactly  what  happened  during 
the  mission.  During  the  replay  the  unit  members  can  observe  the  location,  posture,  and  actions  of  all  the  other 
members.  And,  unlike  live  field  training,  DIVAARS  can  replay  mission  action  exactly  as  viewed  by  any  of 
the  participants.  These  features  not  only  support  the  trainees’  explanation  of  why  events  happened,  but  also 
help  the  unit  members  develop  shared  mental  models  of  individual  and  unit  tasks.  Watching  the  replay  may 
also  strengthen  group  identification  and  cohesiveness.  Finally,  several  DIVAARS  features  (such  as  depicting 
critical  events  in  slow  motion  and  from  multiple  perspectives)  enhance  memory  so  those  lessons  learned  are 
more  likely  to  be  employed  in  subsequent  training  and  missions. 

Figure  8-1  shows  a  sample  DIVAARS  display  with  many  of  these  features  that  can  be  utilized  in  supporting 
an  AAR. 


Figure  8-1:  DIVAARS  Display. 


These  features  are  summarized  below. 

Playback.  A  linear  beginning-to-end  playback  is  unlikely  to  the  most  efficient  way  to  provide  the  trainees 
with  an  understanding  of  what  happened  during  an  exercise.  The  replay  system  includes  actions  such  as  pause, 
stop,  record,  play,  step  forward,  fast-forward,  rewind,  fast-reverse,  and  step-reverse.  Variable  playback  speeds 
are  available.  Furthermore,  the  AAR  Leader  has  the  capability  to  mark  events  during  the  exercise  and  jump 
directly  to  them  during  the  AAR. 

Viewing  Modes.  Viewing  scenario  events  from  different  perspectives  is  essential  to  understanding  what 
happened.  Multiple  viewing  modes  are  available  to  the  AAR  Leader  during  both  the  exercise  and  the  AAR. 
Many  preset  views  can  be  selected  at  any  time  prior  to  or  during  the  exercise  for  immediate  use.  These  can  be 
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used  for  perspectives  or  positions  that  the  AAR  Leader  thinks  will  be  useful,  such  as  the  view  from  an  enemy 
position.  The  variety  of  viewing  modes  provides  added  capabilities  during  the  AAR  process. 

•  Top-Down  -  A  view  of  the  database  looking  straight  down  from  above.  It  can  be  moved  left,  right, 
up,  down,  and  zoomed  in  or  out.  The  AAR  Leader  can  also  lock  the  view  onto  an  entity,  in  which 
case  it  will  stay  centered  directly  above  that  entity  as  it  moves  through  the  database. 

•  2D  View  -  This  is  the  traditional  plan  view  display.  It  is  the  same  as  Top-Down  except  that  depth 
perspective  is  not  shown. 

•  Entity  View  -  By  selecting  any  entity,  (including  enemy  or  civilian),  the  AAR  Leader  can  see  and 
display  exactly  what  that  entity  sees.  This  includes  the  effects  of  head  turning  and  posture  changes. 

•  Fly  Mode  -  The  AAR  Leader  can  “fly”  through  the  database  using  the  mouse  for  control. 

During  the  course  of  a  replay  the  trainees  will  be  able  to  see  the  mission  from  a  number  of  perspectives. 
The  top  down,  2D,  and  fly  views,  views  that  are  never  available  to  them  during  the  mission  exercise,  promote 
seeing  the  big  picture  and  learning  to  see  the  battlefield.  The  entity  view,  seeing  through  the  eyes  of  others, 
supports  a  number  of  training  functions.  Did  the  leaders  see  an  action  or  problem  but  fail  to  respond,  or  were 
they  not  looking  in  the  right  direction  at  all?  Do  squad  members  maintain  360°  security  and  report  promptly? 
What  was  the  view  from  likely  and  actual  enemy  positions?  The  DIVAARS  replay  provides  unequivocal 
answers  to  those  questions. 

Movement  Tracks.  Movement  tracks  show,  in  a  single  view,  the  path  an  entity  travelled  during  an  exercise. 
Markers  are  displayed  at  fixed  time  intervals.  Every  fifth  marker  is  a  different  shape  than  the  four  preceding  it. 
The  display  of  these  markers  can  be  turned  on  and  off.  The  movement  tracks  provide  a  clear  display  of  the 
path  and  speed  of  movement  of  each  member  of  the  unit.  In  addition,  they  provide  indications  of  the  unit 
formations  and  of  the  location  and  duration  of  halts  in  movement.  Thus,  the  AAR  Leader  may  elect  to  skip  or 
fast-forward  through  portions  of  the  replay,  knowing  that  the  movement  traces  for  those  skipped  segments 
will  be  observable  when  the  replay  is  resumed. 

Entity  Identifier.  Because  friendly  force  avatars  in  the  DIVAARS  and  in  the  virtual  simulators  are  not  always 
easy  to  distinguish  from  one  another,  a  unique  identifier  is  shown  above  the  avatar  of  each  unit  member. 
For  example,  2SL  is  the  identifier  for  the  squad  leader,  second  squad.  The  entity  identifiers  change  size  to  be 
readable  across  all  levels  of  zooming. 

Digital  Recording  and  Playback  of  Audio  Program.  Audio  communications  within  a  unit  are  important 
scenario  events.  DIVAARS  records  and  plays  back  audio  content  for  all  scenarios.  This  system  was  developed 
and  tested  with  an  ASTi  Digital  Audio  Communications  System  (DACS:  Advanced  Simulation  Technology, 
Inc.,  2001).  The  ASTi  system  converts  all  voice  communications  from  live  participants  to  digital  messages 
and  outputs  them  on  a  Distributed  Interactive  Simulation  (DIS)  network  using  Transmitter/Signal/Receiver 
Protocol  Data  Units  (PDUs).  In  addition,  DIVAARS  records  environmental  audio  information  (for  example 
gunshots)  from  the  simulator  via  the  DIS  Fire  and  Detonation  PDUs.  The  DIS  timestamps  are  used  to  play 
back  the  audio  at  the  correct  moment  during  the  AAR  replay. 

Viewing  Floors  Within  a  Building.  The  AAR  Leader  must  be  able  to  follow  the  action  in  MOUT  scenarios 
even  when  a  unit  enters  a  building.  The  AAR  Leader  can  select  a  building  and  then  select  a  floor  of  that 
building  to  be  displayed.  Using  this  feature,  the  operator  can  view  and  display  the  avatars  going  through  a 
building  without  the  problem  of  walls  and  upper  floors  blocking  the  view. 
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Bullet  Lines.  This  feature  helps  to  determine  what  objects  are  being  shot  at  by  each  entity,  and  to  identify 
patterns  of  unit  fire.  Bullet  flight  lines  are  shown  for  all  weapon  firings.  The  line  traces  a  shot’s  origin  and 
destination.  It  is  the  same  color  as  the  originating  entity.  These  bullet  lines  gradually  fade  away. 

Event  Data  Collection  and  Display.  DIVAARS  has  the  capability  to  track  many  events  including  shots  fired, 
kills  by  entities,  movement,  and  posture  changes.  These  data  can  be  shown  in  a  tabular  format  or  graphical 
display.  The  AAR  Leader  can  use  them  as  needed  to  make  various  teaching  points.  They  can  also  be  used  to 
support  subsequent  data  analysis  for  research  and  development  applications.  Custom  events  defined  by  the 
operator  are  automatically  flagged  and  can  be  jumped  to  during  playback.  Ten  different  tables  and  graphs  are 
available: 

•  Shots  fired,  by  entity  and  unit; 

•  Kills,  by  entity  and  unit; 

•  Killer- Victim  table  that  shows  who  killed  whom,  with  the  option  to  show  the  angle  of  the  killing  shot 
(front,  flank,  or  back)  or  the  posture  of  the  victim  (standing,  kneeling,  or  prone); 

•  Shots  as  a  function  of  time,  by  entity,  unit,  and  weapon; 

•  Kills  as  a  function  of  time,  by  entity,  unit,  and  weapon; 

•  Kills  by  distance  from  killer  to  victim,  by  entity,  unit,  and  weapon; 

•  Rate  of  movement  of  each  entity,  and  aggregated  at  fire  team  and  squad  levels; 

•  Percentage  of  time  friendly  units  were  stationary; 

•  Percentage  of  time  friendly  units  were  in  different  postures;  and 

•  Display  of  user-defined  events. 

7.1  Evaluation  and  Utilization 

DIVAARS  was  developed  as  part  of  a  comprehensive  program  to  develop  capabilities  for  dismounted 
combatant  virtual  training.  It  was  evaluated  within  the  context  of  the  exercises  conducted  as  part  of  the  overall 
research  program.  In  general,  DIVAARS  has  been  rated  very  highly  by  Soldiers.  Table  8-1  contains  Soldier 
ratings  of  the  systems  capability  to  present  information.  The  ratings  were  collected  as  DIVAARS  was 
developed  and  matured.  The  data  represents  Soldier’s  opinions  drawn  from  a  number  of  different  projects. 
After  its  development  trials  in  2001  and  2002,  DIVAARS  has  been  used  as  the  AAR  tool  in  the  Virtual 
Integrated  MOUT  Training  System  testing  at  Ft.  Campbell,  KY,  in  2004  [10].  It  has  been  used  to  test 
wearable  computer  generated  virtual  Dismounted  Soldier  training  systems  in  2005. 
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Table  8-1:  Ratings  of  DIVAARS  by  Soldiers  in  Dismounted  Soldier  Simulation  Exercises 


The  AAR  system 
made  clear 

Ratings 

2001 

2002 

2004 

2005 

...what  happened 

SA 

44% 

82% 

62%o 

68% 

A 

56%, 

12%o 

31%o 

32%o 

during  a  mission 

Total 

100%, 

94% 

93% 

100% 

...why  things 

SA 

44%, 

76%, 

46%, 

62%, 

happened  the  way 

A 

39%o 

24%o 

35%o 

35%o 

they  did  during  a 
mission 

Total 

83%o 

100%o 

81%o 

97%o 

...how  to  do  better 

SA 

28%o 

71%o 

54%o 

69%o 

in  accomplishing 

A 

56%o 

24%o 

38%o 

23%o 

the  mission 

Total 

84%o 

95%o 

92%o 

92%o 

**&4  =  Strongly  Agree;  A  =  Agree 


It  was  recently  included  in  the  suite  of  capabilities  making  up  the  US  Navy’s  Virtual  Technologies  and 
Environments  (VIRTE)  program.  Within  VIRTE  it  was  used  to  test  methods  for  measuring  Situational 
Awareness  in  virtual  environments.  These  measures  of  Situational  Awareness  may  prove  to  be  a  highly 
effective  means  of  tracking  the  progress  of  units  in  providing  the  intrinsic  feedback  needed  to  support  unit 
performance. 


8.0  CURRENT  AND  FUTURE  RESEARCH  NEEDS 

Changes  in  training  environments,  operational  contexts,  and  operational  systems  has  driven  AAR  research 
resulting  in  new  tools  and  procedures  [17].  Environments,  contexts,  and  systems  continue  to  change,  and  thus 
this  process  of  adapting  AAR  tools  and  procedures  continues.  Networked  command  and  control  systems  and 
joint,  multi-national  operations  are  two  of  the  variables  motivating  additional  AAR  research. 

Networked  command  and  control  systems  enable  new  forms  of  extrinsic  feedback.  Software  can  be  used  to 
provide  feedback  in  mid  exercise  in  the  form  of  intelligent  tutors  [6]  or  “during  action”  review  aids  [1].  These 
software  applications,  if  used  during  actual  operations,  become  sources  of  intrinsic  feedback.  Unit  responses 
to  these  forms  of  feedback  become  a  new  source  of  topics  for  the  AAR,  and  they  also  provide  their  own  AAR 
aids. 

The  AAR  process  may  need  to  be  tailored  to  support  joint  operations,  multi-national  operations, 
and  distributed  training  exercises.  Joint  exercises  include  participants  from  a  mix  of  military  services  and 
multi-national  operations  may  involve  military  and  civilians  representing  a  mix  of  nations  and/or  cultures. 
For  both  joint  and  multi-national  AARs,  cultural  issues  may  influence  the  utility  of  specific  design  features  of 
the  AAR  (e.g.,  is  it  acceptable  for  a  leader  from  another  service,  country  or  culture  to  have  their  mistakes 
revealed  in  front  of  subordinates  or  outsiders?).  In  many  military  training  situations  where  careers  and  prestige 
is  on  the  line,  it  is  possible  that  participants  may  be  more  concerned  with  defending  their  actions  than  with 
learning  how  to  improve  their  performance  in  the  future.  Successful  implementation  of  AARs  under  these 
circumstances  may  require  culture  changes  that  allow  for  open  discussion  of  performance  strengths  and 
weaknesses. 
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9.0  SUMMARY 

Realistic  battlefield  simulations  made  it  possible  for  the  AAR  process  to  replace  the  critique  as  the  primary 
method  of  providing  extrinsic  feedback  after  collective  training  exercises.  Realistic  simulations  provide 
participants  with  intrinsic  feedback  that  cues  and  guides  their  performance  and,  to  some  extent,  let  them  know 
how  well  they  are  performing  various  tasks.  The  intrinsic  feedback  received  by  individuals  depends  upon  their 
job,  their  location  in  the  battlespace,  and  the  quality  of  the  simulation  environment.  This  intrinsic  feedback 
prepares  individuals  to  participate  in  interactive  discussions  that  can  help  a  unit  decide  what  happened,  how  it 
happened,  and  how  to  improve  performance.  A  significant  part  of  the  extrinsic  feedback  process  is  to  bring 
perceived  truth  regarding  exercise  events  in  line  with  ground  truth  (e.g.,  what  actually  happened),  and  the 
sharing  of  intrinsic  feedback  enables  a  view  of  the  situation  that  is  closer  to  ground  truth  than  is  the  view  of  a 
single  individual.  In  general,  less  well  trained  units  will  have  a  greater  need  for  extrinsic  feedback,  because 
they  will  have  less  knowledge  about  the  ground  truth  situation  during  exercises  to  draw  upon.  In  realistic 
battlefield  simulations,  a  wide  variety  of  participants  are  able  to  see,  through  intrinsic  and/or  extrinsic 
feedback,  how  their  actions  contributed  to  the  bottom  lines  of  unit  performance. 

The  AAR  process  makes  use  of  the  Socratic  method  of  asking  leading  and  open-ended  questions  to  guide  unit 
discussions.  The  AAR  process  can  be  expedited  through  the  use  of  aids  that  use  electronic  data  streams  from 
exercises  to  document  key  aspects  of  performance  that  are  close  to  the  root  causes  of  unit  strengths  and 
weaknesses.  Designing  these  aids  and  implementing  their  production  is  a  continuing  activity. 

The  AAR  process  and/or  AAR  aids  have  been  tailored  many  times  to  fit  specific  instances  of  the  live,  virtual, 
and  constructive  training  environments  and  to  fit  changes  in  unit  equipment  and  missions.  This  tailoring 
activity  continues  to  the  present. 
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